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ABSTRACT 

Objectives: Publicly available hospital quality reports 
seek to inform consumers of important healthcare 
quality and affordability attributes, and may inform 
consumer decision-making. To understand how much 
consumers search for such information online on one 
Internet search engine, whether they mention such 
information in social media and how positively they 
view this information. 

Setting and design: A leading Internet search engine 
(Google) was the main focus of the study. Google 
Trends and Google Adwords keyword analyses were 
performed for national and Californian searches 
between 1 August 2012 and 31 July 2013 for 
keywords related to 'top hospital', best hospital', and 
'hospital quality', as well as for six specific hospital 
quality reports. Separately, a proprietary social media 
monitoring tool was used to investigate blog, forum, 
social media and traditional media mentions of, and 
sentiment towards, major public reports of hospital 
quality in California in 2012. 
Primary outcome measures: (1) Counts of 
searches for keywords performed on Google; (2) 
counts of and (3) sentiment of mentions of public 
reports on social media. 
Results: National Google search volume for 75 
hospital quality-related terms averaged 610 700 
searches per month with strong variation by keyword 
and by state. A commercial report (Healthgrades) was 
more commonly searched for nationally on Google 
than the federal government's Hospital Compare, which 
otherwise dominated quality-related search terms. 
Social media references in California to quality reports 
were generally few, and commercially produced 
hospital quality reports were more widely mentioned 
than state (Office of Statewide Healthcare Planning and 
Development (OSHPD)), or non-profit 
(CalHospitalCompare) reports. 
Conclusions: Consumers are somewhat aware of 
hospital quality based on Internet search activity and 
social media disclosures. Public stakeholders may be 
able to broaden their quality dissemination initiatives 
by advertising on Google or Twitter and using social 
media interactively with consumers looking for relevant 
information. 



Strengths and limitations of this study 



Study's novel focus on Internet searches and 
social media mentions to explore modern con 
sumer approaches in gathering and sharing of 
information relating to hospital quality. 
This study's results were limited by its reliance on 
one Internet search engine and on proprietary ana- 
lytical tools. 

Although freely available to researchers, we have 
no way of ensuring that Google's search 
volumes accurately represent Internet searches 
Similarly, our use of a well-accepted commercial 
social media monitoring technology relies on 
proprietary, unverifiable analytics. 



INTRODUCTION 

Public reports of hospital quality are a national 
public health priority at the intersection of 
several important movements in healthcare: 
quality improvement, 1 outcome transparency 2 
and autonomous, patient-centred decision- 
making by informed individuals. 3 Activating 
awareness of, interest in, access to, and increas- 
ing the usefulness of existing public reports on 
hospital healthcare delivery performance can 
positively impact public health. 4 

To achieve these aims, public health stake- 
holders must take account of the mechanisms 
and networks by which information diffusion 
occurs through the new media platforms, par- 
ticularly the omnipresence of Internet search 
engines (eg, Google), 'Web 2.0' (second gener- 
ation Internet-based tools such as Facebook, a 
leading social network) and social media (web 
logs or blogs, microbloggers such as Twitter, 
forums, Internet chat boards and discussion 
groups) . 5 The Pew group notes that one in five 
Americans report that the Internet has helped 
them deal with at least one major life decision 
including health-related decisions. 6 Among 
people who have a significant change in their 
health this proportion reaches one in four. 6 
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The use of online social media and social networks also 
allows consumers to directly reach out to and mobilise 
friends, relatives and experts for help. 6 More than 50% of 
online adults between the ages of 18 and 55 years use 
social networking sites. 7 The use of social media is explic- 
able as part of individuals' well-documented need for 
autonomy, competence and relatedness, intrinsic and 
extrinsic rewards and well-being perceptions. 8 

Despite these new and increasingly popular mechan- 
isms of information dissemination, little research has 
focused on how public health stakeholders can leverage 
such channels to complement existing strategies. In 
theory, such channels could allow fine targeting of 
subsets of consumers with well-defined interests and the 
presentation of relevant health-related information to 
them in cost-effective ways. Such outreach could, for 
example, allow timely matching of preferred providers 
to interested patients. 

It is increasingly clear that merely placing hospital 
quality information on the Internet is insufficient. 9 
Consumers must become aware of the information, they 
ought to have confidence in the information which 
needs to become salient in their mind, allowing the infor- 
mation to become actionable. Media dissemination of 
medical information has been associated with changes in 
consumer behaviour. A significant drop in vaginal births 
after caesarean section (CS) followed reports of increased 
risk in the traditional media and on the Internet. 10 
Before the Internet age, there was a significant increase 
in faecal occult blood tests following the widespread 
media attention to President Reagan's colon cancer. 11 

However, the public (and referring physicians) appear 
to underuse online hospital quality reports. For 
example, the introduction of Hospital Compare, 
Medicare's public reporting initiative, was not associated 
with reductions in mortality beyond existing trends for 
acute myocardial infarction. 12 Similarly, public reporting 
of provider and hospital CS rates via an interactive 
website in Virginia since 1996 has not controlled CS rate 
growth — indeed the rate of CS deliveries has risen faster 
in Virginia than in the USA as a whole. 13 

This shortfall between the promise of public reports 
in informing and empowering consumers to choose 
higher quality, safer or more preferred providers and 
their limited success in achieving such shifts in con- 
sumer choice in practice has been attributed to design 
and formatting, 14 relevance and appropriateness. 15 
Others argue that methodological issues are import- 
ant, 16 that some online behavioural interventions have 
been effective, or that approaches for reporting quality 
information have been slowly evolving towards effective 
strategies that help consumers. 18-21 

However, an additional explanation may be that most 
consumers do not know that such information exists 
and/or that it could be helpful for them. Were this to 
be true, then one might expect few searches for such 
information on the Internet, and few mentions of such 
information or its sources on social media. 



To our knowledge, no previous study has used 
Internet and social media search tools to understand 
whether consumers are actually aware of publicly avail- 
able hospital quality reports, have searched for them, 
and have discussed these with others. Such tools could 
allow public health researchers to understand which 
entities produce and disseminate reports that have 
higher awareness or have been searched for more often. 
While such awareness is a necessary step in driving con- 
sumer activation higher it is clearly not sufficient, and 
our study did not investigate whether consumer behav- 
iour changed as a result of searches or mentions of 
quality. 

We used Google's in-house search and Adword analy- 
tics tools to understand the volume and types of hospital 
quality and quality report searches occurring through 
Google's market-leading Internet search engine. Google 
has maintained an approximate two-thirds of the market 
for Internet search, ahead of Yahoo and Bing. 22 We also 
used a proprietary social media monitoring tool to inves- 
tigate all instances in California in which any of six 
major publicly available hospital quality reports were 
mentioned on social and traditional media, and the atti- 
tude of users (their 'sentiment'). 

Although consumer awareness of such information is 
clearly necessary for it to inform consumer decision- 
making, however it is insufficient. Our study was not 
able to infer whether Internet search activity or social 
media mentions were associated with the processing of 
such information. We also did not analyse whether the 
information contained in the publicly available hospital 
quality reports was reliable, consistent, relevant and 
actionable. Moreover, our study was fundamentally 
limited by a focus on just one Internet search engine, 
Google. We ignore searches occurring on Yahoo and 
Microsoft's Bing, and also do not ascertain important 
other channels of accessing such information. For 
instance, we do not know how much direct Internet 
access occurs from consumers to a provider website, or 
how many individuals click through an advertisement on 
one site directly through to a particular hospital or 
public sector website on healthcare quality. 

Despite these limitations, the findings in this study 
may still serve to inform public dissemination strategies. 
Beyond this, our presentation of advanced Internet 
search engine and social media analytical tools may 
allow other researchers to add such analyses to their 
own, unrelated studies. 



METHODS 

Internet search volumes 

To understand Internet search behaviour, we focused on 
the market-leading search engine, Google. We used two 
Google tools to understand Google search engine 
volumes. Google's business model depends on online 
users seeing or 'clicking through' display advertisements. 
These advertisements appear adjacent to search results 
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and next to content on third party websites. Relevant 
advertisements are more likely to be viewed and clicked 
through, so Google's revenue depends on accurately 
matching advertisement keywords with those contained 
in searches or in published content elsewhere. 

Google provides two research tools to investigate key- 
words and search terms. Both tools are free to researchers 
after the creation and registration of a user account. We 
first surfaced the most common search terms in the last 
year relating to three so-called seed terms', "hospital quality", 
"best hospital" and "top hospital" using Google Adwords' 
Keyword Planner. 23 The degree of relationship is calcu- 
lated by Google using proprietary algorithms incorporat- 
ing similar user search and purchasing behaviour. For 
example, Google suggests that "patient safety" is a related 
keyword to "hospital quality". However, AdWords did not 
surface "healthgrades" as a keyword related to our three 
seed terms of "hospital quality", "top hospitals" and "best 
hospitals". Google AdWords' Keyword Planning Tool is 
designed primarily to suggest keywords for advertisers 
seeking to create matches between their own online 
display advertisements and others' online content. 
Consequently, advertisers are barred from using a different 
entity's trademarks in their own keyword list. 24 

From the suggested keywords we then narrowed down 
searches based on an arbitrary threshold of 1000 searches 
per month and by our determination of their specificity to 
hospital quality (see table 1). We then used the Google 
Trends tool to quantify relative search volume by term, 
and by state, over the 12-month period from. 25 

Social media disclosures 

To understand instances ('mentions') in which hospital 
quality terms and hospital quality reports were discussed 
in social and traditional media, we used a proprietary 
social media monitoring tool (MAP) from Marketwire/ 
Sysomos (http://www.sysomos.com). Sysomos has mainly 
corporate customers which include Microsoft, Coca-Cola, 



Google, Marriott, Nike and other leading consumer 
good companies. 

This tool was leased by us as a 'software-as-a-service', 
and allowed us to remotely access a database maintained 
by Sysomos to investigate hundreds of millions of current 
and historical data sources from all publicly disclosed 
blog, forum, social and traditional media sources (text, 
image and video) originating in California between 1 
January 2012 and 20 December 2012 for the six public 
hospital quality reports listed below. 

Social media tool queries were constructed using a 
Boolean structure to limit extraneous mentions (eg, 
reuters reporting on a hospital construction) and to 
capture abbreviations (eg, USNWR). In addition to men- 
tions, several other commonly used marketing metrics 
were tracked by MAP. 

'Reach', the degree of authoritativeness of the online 
content or user, was estimated by MAP using a proprietary 
algorithm. Reach is summarised for each mention as a 
score between 0 and 10 based on its authority and influ- 
ence. This measure is based on a number of different 
factors as to assess the influence and reach of the media 
source. For example, if the mention is on a blog, the 
measure incorporates posting frequency, number of 
unique links into and out of the blog, and number of 
follow-up comments. 'Impressions' are an estimate of the 
number of total views of content seen and were similarly 
estimated by Sysomos' proprietary analytics. 

Finally, sentiment was analysed by MAP's proprietary 
natural language processing (NLP) software. This is 
again carried out automatically by MAP using proprietary 
algorithms, which summarise the sentiment of a set of 
such writings (eg, all social media writing regarding 
Healthgrades, say) by what proportion was positive, nega- 
tive or neutral. Sysomos claims an 85% overall match rate 
between its MAP classification algorithms compared to 
gold standard manual (ie, human expert) categorisation. 

Formally, sentiment analysis is the use of NLP tools to 
identify, extract, grade and categorise subjective 



Table 1 Most popular Google searches for terms relating hospital quality, 1 August 2012 to 31 July 2013 



Exact Google search term 



Monthly search volume 



Exact Google search term 



Monthly search volume 



hospital compare 1 2 1 00 

doctor reviews 9900 

best medical schools 9900 

doctor ratings 6600 

core measures 2900 

rate doctors 2400 

patient safety 2400 

hospital ratings 2400 

customer satisfaction survey 2400 

best hospitals in the us 2400 

us news best hospitals 1 900 



best hospitals 1 900 

top hospitals in the us 1 600 

patient satisfaction 1 600 

hospital rankings 1 600 

patient satisfaction survey 1 300 

best med schools 1 300 

best cancer hospitals 1 300 

physician ratings 1 000 

nursing home ratings 1 000 

best hospitals in the world 1 000 



Monthly search volume averaged over 12 months preceding 31 July 2013. Twenty-one most popular national Google searches related to "top 
hospital", "best hospital" and "hospital quality" seed terms as determined by Google Adwords Keyword planner, excluding terms with fewer 
than 1000 monthly searches or terms not specifically or exclusively related to quality in the authors' determination. See supplementary 
material for list of 75 search words before exclusions. 
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information (eg, attitude of blogger to the content 
being discussed, mood of the person sharing some infor- 
mation on social media) contained within the writing. 
NLP tools effectively allow computers to understand 
human language. 26 Similar tools have been used by 
health services researchers to assess the effectiveness of 
labelling adverse drug events, 27 to mine patient medical 
record free text for evidence of diabetes care 28 and to 
ascertain patient falls faster than incidence reports by 
applying NLP on imaging requests. 29 

Publicly available hospital quality reports 

Publicly available hospital quality reports in California 
included the following six major sources. From the infor- 
mation and news industry we selected Thomson Reuters' 
Best Hospitals ranking (http://www.lOOtophospitals. 
com), US News & World Report's ranking of top hospi- 
tals (http://health.usnews.com/best-hospitals) and 
Healthgrades, a large private supplier of national phys- 
ician and hospital quality information (http://www. 
healthgrades.com/) . 

From government sources we used Medicare's 
Hospital Compare tool (http://www.medicare.gov/ 
hospitalcompare/) and California's state Office of 
Statewide Healthcare Planning and Development 
(OSHPD) reports (http://www.oshpd.ca.gov). Lastly, we 
selected the non-profit CalHospitalCompare (http:// 
www.calhospitalcompare.org). Hospital Compare, 
Thomson Reuters and US News hospital quality reports 
were not specifically related to maternity care. 

RESULTS 

Internet search volumes 

We explored search volume for popular terms related to 
'top hospital', 'best hospital' and 'hospital quality'. We 
collected the intersection of all 2095 suggested keywords 
related to these three seed terms which had been 



surfaced by Google AdWords Keyword Planning tools. 
We then excluded search terms with less than 1000 
monthly searches nationally (full list available on 
request) . 

From the resulting 75 search terms comprising 
610 700 searches nationally each month on average, we 
then further excluded search terms relating to specific, 
named hospitals (eg, "dana farber cancer institute"), 
geographies (eg, "hospitals in los angeles") as well as 
search terms not obviously related to quality (eg, "hos- 
pital gowns", "hospital beds") or not exclusively related 
to quality (eg, "hospital", "medicare"). 

This process yielded 21 search terms (table 1) compris- 
ing a total of 97 700 searches nationally per month on 
average. We focused further analysis on the most 
common search terms within this list. We used another 
tool, Google Trends, to track national and Californian 
online search activity on Google's search engine for 
terms related to hospital quality and quality reports 
between 10 August 2012 and 2 August 2013. Google 
Trends results are normalised by Google so that the peak 
search volume over the terms analysed is set to 100. 25 

In figure 1, we show national time trends for searches 
for the four most popular hospital-related terms as per 
table 1. These were "hospital compare" (presumably 
intended as a search for the Federal Government's 
Hospital Compare website), "hospital ratings", "best hos- 
pitals in the us" and "us news best hospitals". 

Nationally, searches for the Hospital Compare term 
dominated those relating to the other search terms, 
except in mid-July, where the contemporaneous publish- 
ing of US News and World Report's rankings occurred. 
Interestingly, there was also a transient spike in search 
volume for "hospital ratings" at the same time, although 
not for the other two search terms. 

In unreported analyses we examined search volumes 
for terms that focused specifically on physicians not hos- 
pitals, despite being algorithmically related to the 



Figure 1 Google National 
Search Trends for Hospital 
Quality and Reports, 1 August 
2012 to 31 July 2013. Note: 
search volume data provided by 
Google Trends for national 
searches. Four terms represented 
the four most common keywords 
surfaced by Google AdWords 
related to seed terms "hospital 
quality", "best hospitals" and "top 
hospitals". Transient search 
increase in mid-July 2013 for 
"US News Best Hospitals" and 
"Hospital Ratings" coincides with 
release of US News' 2013-2014 
Best Hospitals rankings. 
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hospital-centric seed terms according to Google 
AdWords. We found that "doctor reviews" and "best 
medical schools" dominated "doctor ratings", and that 
national searches for all three terms were carried out 
throughout the year with greater variability than the 
hospital-related searches shown in figure 1. Within 
California, there were insufficient searches for "doctor 
ratings" to be reported by Google. 

We also compared search volume in figure 2 between 
the leading not-for-profit publicly available hospital 
quality reports (Hospital Compare) and the leading for- 
profit one (Healthgrades) . We found that "healthgrades" 
and its mis-spelled version "health grades" were each 
more commonly searched for than "hospital compare", 
and that search volumes varied substantially over time. 
We also observed a very large, transient pair of spikes in 
search volume in mid-January 2013 for "healthgrades" 
and the mis-spelled variant which coincided with release 
of Healthgrades' Top Hospitals for Outcomes list. 

We disaggregated national searches into state-level 
searches, data not shown, and note in passing an unex- 
pectedly varied distribution geographically for many 
terms, and these differences did not remain constant 
across the search terms. 

Social media disclosures 

Using the commercially available MAP software, we 
examined a complementary aspect of consumer aware- 
ness and interest. MAP tracked mentions of any of the 
public quality reports in consumer blogs, in comments 
on forums, and in social media more generally. In 
table 2, we summarise this measure of consumer activa- 
tion for each of the six reports in 2012, focusing only on 
consumers self-identifying as originating in California. 

Generally, there were few direct mentions of any of 
these reports on either social media or traditional media, 
with less than 1000 instances in each type of media. In 
unreported analyses, the typical posters of such content 



were healthcare providers, academics, public health 
agencies and commercial entities and stakeholders in the 
healthcare system. The estimated number of online users 
who were exposed to these mentions in aggregate was 
large for users of Twitter (>75 000 individuals) and for 
traditional media (>275 000) but small or not estimated 
by MAP for the other social media platforms. 

Irrespective of social, online or traditional media type, 
the three commercial suppliers of hospital quality infor- 
mation appeared to dominate consumer awareness 
using this research tool. The federal government's 
Hospital Compare lookup tool was a distant fourth in 
mentions across all platforms, but in turn overwhelmingly 
dominated the state-level OSHPD and the non-profit 
CalHospitalCompare reports which were effectively absent 
from publicly disclosed social conversations. 

To classify the sentiment expressed towards the reports, 
we used MAP's proprietary sentiment analyses to gauge 
the disposition of consumers to these reports. Twitter sen- 
timent is typically 'neutral', given the short length. In 
blogs and traditional news media sources, reaction to 
content was generally highly positive for Hospital 
Compare, Healthgrades, US News and World Report's 
Top Hospitals and Thomson Reuters 's Best Hospitals 
public quality reports. Given the absent mentions of the 
OSHPD and CalHospitalCompare hospital quality 
reports, MAP was unable to determine sentiment. 

DISCUSSION 

Our exploratory study used on market-leading Internet 
search engine, Google and Google Trend search tools to 
show substantial Internet search activity broadly relating 
to hospital quality, exceeding half a million monthly 
searches nationally with a more narrowly defined set of 
nearly one hundred thousand monthly searches. To put 
this into context, a high traffic health-related website 
such as WebMD is searched for on Google more than 1.8 
million times per month (and additional inquiries are 
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Table 2 Social and traditional media mentions and sentiment for hospital quality reports: California 1 January 2012 to 20 
December 2012 



Commercial hospital quality reports 



Media 



Metric 



Thomson 
Reuters best 
hospitals 



US News & 
World Report 
Healthgrades top hospitals 



Federal State Non-profit 

Hospital Cal Hospital 

compare OSHPD compare 



Social media 



Blogs 



Traditional media 



Twitter 


Mentions 




1 A 
I H- 


1 i 
I I 


0 

c. 


u 


u 




Impressions 


UU/ 


0 i oH-y 


I O / O I 




u 


u 




Sentiment 


+ 0/0 


no/ 

U /o 


no/ 

U /o 


no/ 

U /o 


MA 


MA 
INM 






^ 070/ 

^ y/ /o 


I UU /o 


1 nno/ 

I UU /o 


1 nno/ 

I UU /o 


MA 


MA 
INM 






— U /o 


no/ 

U /o 


no/ 

U /o 


no/ 

U /o 


MA 


MA 
INM 


Facebook Mentions 


n 
u 


0 


u 


u 


U 


U 


Google+ 


Mentions 




OA 


R7 

0/ 




u 


u 




Sentiment 


+ /o 


I 0 /o 


OH- /o 


1 O /o 


MA 
INM 


MA 
INM 






^ U«J /o 


00 /o 


\JC /o 


7«% 
#0/0 


MA 


MA 






70/ 
— 1/0 


A©/ 


A©/ 
H- /o 


Qo/ 

y /o 


MA 
INM 


MA 
INM 


Forum 


Mentions 


u 


O 
C. 


D 


I 


U 


U 




Impressions 


U 




D 


-4 

1 


U 


U 


Youtube 


Items 


95 


86 


2232 


21 


4 


0 


Video 


Items 


22 


12 


107 


3 


0 


0 




Mentions 


674 


19 


22 


23 


2 


0 




Reach 


4.0 


4.8 


3.0 


6.0 


1.5 


0.0 




Sentiment 


+ 100% 


80% 


91% 


80% 


0% 


NA 






~ 0% 


20% 


9% 


20% 


100% 


NA 






- 0% 


0% 


0% 


0% 


0% 


NA 




Mentions 


512 


318 


156 


87 


13 


0 




Impressions 


274 341 


2009 


957 


1290 


283 


0 




Sentiment 


+ 97% 


97% 


98% 


87% 


100% 


NA 






~ 3% 


2% 


1% 


13% 


0% 


NA 






- 0% 


1% 


1% 


0% 


0% 


NA 



Mentions (specific instances of public hospital quality reports mentioned in media), reach (authority level score out of 10), impressions (total 
individuals exposed) and sentiment (content attribute) for the period 1 January 2012 through 20 December 2012, except for Facebook which 
is limited to status updates in the 6 weeks prior to 20 December 2012. MAP query terms were {"Healthgrades" AND "quality"}, {"OSHPD" 
AND "quality"}, {("Calhospitalcompare" OR "Cal hospitalcompare" OR "Calhospital compare" OR "Cal hospital compare") AND "quality"}, 
{("ThomsonReuters" OR "Thomson Reuters") AND "Hospital"}, {("Hospitalcompare" OR "Hospital compare") AND "quality"}, and {("US News 
World Report" OR "USNWR") AND "Hospital"}. 
OSHPD, Office of Statewide Healthcare Planning and Development. 



made directly by typing webmd.com into a browser, cir- 
cumventing a search engine). Our main rinding was that 
commercially produced hospital quality reports were 
searched for more commonly than government reports, 
although searches for HospitalCompare dominated 
other health-related quality keywords. Unsurprisingly, 
state-based variation broadly reflected number of consu- 
mers but variation differed by keyword searched for on 
Google. 

We supplemented this analysis of awareness by examin- 
ing social media activity using a different, social media 
monitoring research tool. This analysis revealed little 
social media and social network references to hospital 
quality reports generally, and the instances found tended 
to be from providers, experts, regulators, academics and 
not individual consumers. However, of the little activity 
documented, we again found that commercially pro- 
duced hospital quality reports were more often men- 
tioned than not-for-profit or government ones. 

Our study has several strengths. Our focus was both 
nationally and on the racially and ethnically diverse, 



large state of California which suggests that the results 
may have external validity. Yet, our study also has several 
important limitations. 

Most importantly, our study's ascertainment of con- 
sumer access to healthcare information is fundamentally 
limited to just one firm operating in one such channel. 
Within the space of Internet search engines, our focus on 
Google excluded Microsoft's Bing, Yahoo and other 
smaller but still important Internet search engines. 
Moreover, we have little understanding of how Google's 
users might differ from those of other search engines. 
Outside this space, our study was not designed to ascer- 
tain individual's use of, for example, advertisements in 
web articles (with direct clickthrough to a source of 
quality information) or the direct use of their Internet 
browser (to directly enter an already known website such 
as healthgrades.com). Thus, our study's results in the 
space of Internet search engines should be seen as a 
lower bound only. 

A separate limitation is that our social media analyses 
were procured through the use of MAP, an analytical 
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software-as-a-service from Marketwire/Sysomos a leading 
social media monitoring technology provider. This service 
uses proprietary non-transparent NLP algorithms and 
search methods. It is possible that company-specific 
design decisions and negotiated upstream access to social 
media platforms may have affected the reported results. 

Similarly, our Internet search analyses were performed 
using Google's proprietary research tools, and we have 
no ability to confirm that the data supplied by Google 
accurately or completely represent all searches. However, 
we have no reason to believe that the data were masked, 
distorted or otherwise manipulated by Google in ways 
that might have biased the results. To reiterate, we have 
little understanding of how representative Google users 
are of Internet search engine users in general, or even 
of Internet users using direct entry of website addresses 
into their browser or clicking through from ads directly 
to quality information sites. Despite Google's command- 
ing share of the market, estimated at two-thirds, 22 future 
work will address the extent of sample selection that 
could arise from focus on a single search engine. Finally, 
we have assumed without data that an individual search- 
ing for 'health grades' is interested in quality and search- 
ing for this reason. This assumption may be wrong, for 
example, as Healthgrades also informs users as to closest 
hospital to a particular zipcode or geographical area. 
Without data on intent, our assumption may therefore 
lead us to overstate interest in quality. 

Many stakeholders have the objective to increase con- 
sumer awareness of hospital quality reports, and use of 
the Internet to disseminate such reports is common, 
increasing and promising. Yet, the largest and most 
recent study showed an apparent limited effectiveness of 
the flagship Hospital Compare reports in reducing mor- 
tality for key conditions requiring hospital admission. 12 
This shortfall between the promise and the actual has 
been attributed to many causes. In this study, our focus 
was on whether consumers know that such information 
exists and/or that it could be helpful for them. 

Our use of Internet and social media search tools in 
this exploratory study suggests that consumers are some- 
what aware of publicly available hospital quality reports, 
have searched for them and have discussed these with 
others. 

We believe that these results suggest that there may be 
some lessons that could be learned by public stake- 
holders from the private sector. Such lessons could 
further their objectives of the enhancement of quality, 
greater transparency and more empowered consumers. 
To bridge the gap between the promise and the actuality 
of hospital quality reports, marketing approaches and 
consumer activation approaches such as widespread 
advertising might help 'get the word out'. Despite sub- 
stantial focus group testing of report contents and 
formats at CalHospitalCompare, 30 for example, our 
study finds that their public reporting efforts appear not 
to have resonated with consumers after their initial 
launch in 2007. 



Fine-tuning of existing public quality reports to 
enhance their visibility and usability may also help, as 
could a greater understanding of the crowded field of 
ratings-producing entities and the limited attention span 
of consumers. Understanding how consumers see, react 
to and use the information from all players in this space 
— private and not for profit — could allow organisations 
to benchmark their performance, tailor and distinguish 
their offering from others' in the market. 

Many of these efforts may require substantial invest- 
ments. We estimate that Healthgrades spends in excess 
of US$50 million per year in producing, hosting and 
advertising its services. Replicating a similar approach 
would be infeasible for a county, but might be possible 
for a larger state health department. Such entities could 
advertise relatively cheaply on Google's search engine, 
paying for 'clicks' through to their own websites by indi- 
viduals who search for hospital quality-related terms. 

Many of the keywords in table 1 are priced at around 
US$1 per click at Google and on other search engines: 
an advertiser only pays this amount if the online user 
clicks the advertisement and navigates through to the 
advertiser's website. For example, a budget can be set in 
advance, limiting financial risk for a small agency. 
Similarly, cheap advertising can be purchased through 
sponsored tweets on Twitter. Certainly, at the federal gov- 
ernment level such levels of expenditure are easily pos- 
sible and might assist in elevating awareness and use of 
Hospital Compare. 

Cheaper niche marketing efforts in building con- 
sumer awareness could also include engagement strat- 
egies using social media instead of other forms of 
traditional advertising or passive Internet website cre- 
ation. Examples of campaigns on social media that have 
succeeded through identification of leads and direct 
interaction with online users span the education indus- 
try, entertainment, small business and many other 'viral 
marketing' settings. 31 Stakeholders could exploit the 
advertising platforms on, for example, Facebook to serve 
advertisements to finely targeted (through Facebook's 
extensive private knowledge of their users interests and 
private disclosures) segments of individuals 'in the 
market' for a particular health-related message. 

The use of social media is increasingly part of indivi- 
duals' well-documented need for autonomy, competence 
and relatedness, intrinsic and extrinsic rewards, and well- 
being perceptions. 8 In addition, by virtue of its open 
source nature, social media enables much more efficient 
diffusion of information, network 'cascades' and perme- 
ation of many more social network. 32 In contrast, the 
traditional highly passive view of online consumers indi- 
vidually receiving broadcast messages from a static 
website and acting on them has shown at best only 
modest evidence of success in practice. 14 15 

Finally, a focus on more narrow subsets of customers, con- 
ditions or geographies may also be the optimal use of 
scarce public health resources. Focusing on the most 
important set of 'shoppable conditions' rather than trying 
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to cover every area of healthcare quality may be a more 
cost-effective strategy Examples of such conditions with 
high variation in quality include maternity care. 33-37 
Targeted engagement at women of childbearing age, 
women who are pregnant and their family and friends 
would allow a focus on foreseeable delivery healthcare 
needs. 

Both social media interaction (eg, through a tool like 
Sysomos' MAP which allows real time identification of 
consumers posting relevant content on social media) 
and Internet search-linked advertising (eg, through 
Google's AdWords which allow fine delineation of what 
sort of individuals, where, etc, see advertisements) are 
feasible tools to reach such niche groups. Such tools 
could be fruitfully and potentially cheaply used to 
educate mothers regarding which hospitals in her neigh- 
bourhood have consistently lower primary CS rates, 
higher vaginal birth after cesarean (VBAC) rates, greater 
lactation consultation access or higher breastfeeding 
rates. 

CONCLUSION 

A strong alignment of interests in stimulating quality 
improvement, increasing accountability and improving 
consumer choices appears increasingly opportune for 
public health stakeholders. An increasing variety of 
cheap technology tools allow stakeholders to assess and 
drive consumer awareness of healthcare choices. Given 
the importance of hospital quality information to 
payors, providers, consumers and regulators, we are opti- 
mistic that the science of public reporting will continue 
to develop in the ways outlined here. 
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